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ABSTRACT 

Virtual environments are computer-generated synthetic environments with which a human user 
can interact to perform a wide variety of perceptual and motor tasks. At present, most of the virtual 
environment systems engage only the visual and auditory senses, and not the haptic sensorimotor 
system that conveys the sense of touch and feel of objects in the environment. Computer keyboards, 
mice and trackballs constitute relatively simple haptic interfaces. Gloves and exoskeletons that track 
hand postures have more interaction capabilities and are available in the market. Although desktop and 
wearable force-reflecting devices have been built and implemented in research laboratories, the current 
capabilities of such devices are quite limited. To realize the full promise of virtual environments and 
teleoperation of remote systems, further developments of haptic interfaces are critical. 

In this paper, the status and research needs in human haptics, technology development and 
interactions between the two are described. In particular, the excellent performance characteristics of 
Phantom, a haptic interface recently developed at MIT, are highlighted. Realistic sensations of single 
point of contact interactions with objects of variable geometry (e.g., smooth, textured, polyhedral) and 
material properties (e.g., friction, impedance) in the context of a variety of tasks (e.g., needle biopsy, 
switch panels) achieved through this device are described and the associated issues in haptic rendering 
are discussed. 


'Part of the work reported here was supported by Naval Training Systems Center Contract No. N61339-94-C-0087. 



APPLICATIONS OF SYNTHETIC ENVIRONMENTS (SE) 


Synthetic environments (SE), which include both virtual environments (VE) and teleoperation, 
have generated considerable excitement, owing to the wide variety of applications in which they can 
play a significant role (listed below; Ref. 1). At present, most of the VE systems engage only the 
visual and auditory senses, and not the haptic sensorimotor system that conveys the sense of touch and 
feel of objects in the environment. Manual interactions with SE are important in sensorimotor tasks 
such as training of surgeons with VE or conveying the feel of an object to the participants of a 
teleconference. They may also play a significant role in cognitive tasks such as memorization and 
analysis of multidimensional databases or teaching the implications of the violations of physical laws. 


• Design, Manufacturing, and Marketing 

• Medicine and Health Care 

• Teieoperation for Hazardous Operations 

• Training 

• Education 

• Entertainment 

• Information Visualization 

• Telecommunications 

Manual interaction with SE is important in 
particular tasks within each application area. 


Figure 1 
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HAPTICS: MANUAL INTERACTIONS WITH THE ENVIRONMENT 
- FOR EXPLORATION OR MANIPULATION 


The term Haptics refers to manual interactions with real or virtual environments. It includes 
both exploration of objects to obtain information about the environment and manipulation of objects to 
alter the environment. In contrast to the purely sensory nature of vision and audition, the human haptic 
system involves tight integration of both sensory and motor components. Further, the sensory 
information can be divided into two classes: (1) tactile information, referring to the sense of contact 
with the object, mediated by the responses of low-threshold mechanoreceptors innervating the skin 
(say, the finger pad) within and around the contact region; and (2) kinesthetic information, referring to 
the sense of position and motion of limbs along with the associated forces, conveyed by the sensory 
receptors in the skin around the joints, joint capsules, tendons, and muscles, together with neural 
signals derived from motor commands. 


Haptics: Manual interactions with the environment 
- for exploration or manipulation 



Sensory 


Motor 


Tactile 


Kinesthetic 

(proprioceptive) 


Figure 2 
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HAPTIC INTERFACES 


Haptic interfaces are devices that enable manual interaction with virtual environments or 
teleoperated remote systems. They are employed for tasks that are usually performed using hands in 
the real world, such as manual exploration and manipulation of objects. In general, they receive motor 
action commands from the human user and display appropriate tactual images to the human. The 
command and display variables are listed below. 


Haptic Interfaces 



CbmrHnd 
(motor actions) 

Fbsture/nrtion -*■ 

or 

Contact force 


Display 



-►Contact force 
or 


Tactile display 


►Fbsture/nrjtion 


Thermal, 
electrical, etc. 


Force distribution 


(spatial-tenpora!) 


Figure 3 




CLASSIFICATION OF HAPTIC INTERFACES 


Haptic interfaces can be classified in several ways. First, classification is based on whether 
they are force-reflecting or not, as well as by what types of motions (e.g., how many degrees of 
freedom) and contact forces they are capable of simulating. The second type of classification is based 
on whether they are simulating the touch, feel, and manipulation of objects directly in contact with the 
skin or through a tool. A third set of important distinctions are based on whether the force display 
systems are ground-based, such as joysticks and other hand controllers, or body-based, such as gloves 
and exoskeletons. 


1 . Based on motions and/or forces (e.g., 
presence or absence of force reflection, 
degrees of freedom, types of forces) 

2. Ideal exoskeleton or tool handle approach 

3. Ground-based or body-based 


Figure 4 
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AVAILABLE HAPTIC INTERFACES 


A variety of haptic interfaces are currently available. Computer keyboards, mice and trackballs 
constitute relatively simple haptic interfaces. Gloves and exoskeletons that track hand postures have 
more interaction capabilities and are available in the market. Although desktop and wearable force- 
reflecting devices have been built and implemented in research laboratories, the current capabilities of 
such devices are quite limited. There exist a number of examples of tactile stimulators for the finger, 
including pneumatic shape changers, electrocutaneous stimulators, and vibrating arrays, but none 
provides convincing tactile images and all are awkward to use (Ref. 2). 


• Position sensors 

• Joysticks 

• Point-interaction robotic devices 

• Teleoperator masters 

• Exoskeletal devices: 

- flexible (gloves and suit worn by user) 

- rigid links (joined linkages affixed to user) 

• Tactile displays: 

- shape changers (shape memory actuators, 

pneumatic actuators, micro-mechanical 
actuators) 

- vibrotactile 

- electrotactile 


Figure 5 


SOFTWARE FOR HAPTIC INTERACTIONS 


Similar to the software needed to generate visual images, the software necessary to generate 
tactual images can be classified into three major groups: haptic interaction software, simulation of 
object behavior, and software for rendering tactual images. Haptic interaction software mainly consists 
of reading the state of the haptic interface device. Simulation of object behavior requires physical 
models of virtual objects. This can be accomplished either by a unified model for all the modalities 
(e.g., visual, haptic, acoustic) or through separate models for each modality, together with correlation 
algorithms for consistency among the displays corresponding to each of the modalities. The software 
for rendering the tactual images receives the output of the physical model and generates the commands 
needed to drive and control the interface device. 


Software for Haptic Interactions 


Device State - Reading and interpreting the state of 
the haptic devices. 


Simulation 


, Unified model of all modalities 


* Modality -specific models 


Rendering - Control of haptic display 


Figure 6 


no 



THE PHANTOM 


The Phantom is a force-reflecting haptic interface recently developed at MIT (Ref 3) It is 
capable of generating realistic sensations of single point contact interactions. It is essentially a robot 
with six degrees of freedom, capable of generating a three-dimensional force vector at its end effector 
which can either be a thimble or a stylus that is manipulated by the human user. 



Figure 7 
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SLIDER SWITCHES 


A variety of touch interactions have been haptically rendered using the Phantom (Ref. 4). 
Shown below are three types of slider switches and their respective force-displacement behavior. In 
addition to the properties of mass, viscosity, stiction, and surface stiffness, each cube has an 
underlying characteristic spring function. 


Slider Switches 



In addition to the properties of mass, viscosity, 
stiction, and surface stiffness, each cube has an 
underlying characteristic spring function. 


Figure 8 
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BUTTONS WHICH "CLICK" 


The feel of push buttons which "click" has also been displayed through the Phantom to the 
human user. Each of the buttons shown below simulate the force-displacement relationship shown 
schematically, but feel distinct owing to differences in the values of parameters such as stiffness. 


Buttons which ’’Click” 



Figure 9 
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RENDER: RENDERS POLYGONAL SURFACES 


The haptic display of arbitrarily shaped objects represented as polyhedra has been achieved with 
the Phantom. The rendering software uses standard graphics file formats and is capable of simulating 
both convex and concave surfaces. 



• Allows haptic display of arbitrarily shaped 
objects 

• Uses standard graphics file formats 

• Convex and concave 

• Can render arbitrarily thin objects 


Figure 10 
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Two Phantoms have been used together to allow a user to perform two-fingered manipulation 
of virtual objects as shown below. Such contact interactions are analogous to the use of tools in real 
environments. 



Figure 1 1 


115 




BLOCKS: DYNAMIC SIMULATION OF BLOCKS IN 3D 

Using two Phantoms, the dynamic simulation of 3D blocks being manipulated by a user has 
been achieved. Both visual and haptic displays were provided to the user. 



• Two hand/finger manipulation 

• Static friction model 

Phantom and walls, Phantom and blocks, 
blocks and blocks 

• Gravity 

Figure 12 


116 



WHAT ARE ELEMENTS OF HAPTIC INTERACTION? 


Listed below is a summary of the elements of haptic interactions. 


• Sensed elements: 

Motion, force, tactile, temperature, 
heat flow, current flow, pain, etc. 

• Perceived events and states: 

Impact 

Sustained contact 
Slip 

Friction, texture 
Freedom/constraint in motion 
Compliance 
Curvature 

• Workless interactions: 

Imposing and detecting constraint 

• Work interactions: 

Force over distance, impulse, 
momentum and energy 


Figure 13 
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WHAT WE HAVE DONE 


The status of the work done at the MIT AI Laboratory is given below: 


• Developed class of haptic interface 
permitting force vector display - Phantom 

• Demonstrated basic interaction elements 
impact and constraint forces 

object shape, motion 
friction, texture 

surface and object, impedance 

• Combined basic elements to build simple 
mechanical worlds: 

astroid, blocks, 
needle-biopsy, switch panels 

• Developed rendering algorithms for 
polyhedral objects 

• Begun development of rendering algorithms 
for visco-elastic materials 


Figure 14 
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HUMAN HAPTIC PERFORMANCE 


A basic understanding of the biomechanical, sensorimotor, and cognitive abilities of the human 
haptic system is essential for further improvements in the design of hardware and software of haptic 
interfaces. Summarized below are some of the quantitative data (Refs. 2 and 5) on human haptic 
performance which set some of the design specifications of interface devices. 


Tactile Sensory System: 

Vibrations - detectable up to Ikhz 

thresholds - 0.3 to 30 microns 

Spatial resolution - 1mm at fingerpad 

Kinesthetic Sensory System: 

Just noticeable differences (JND) for joint 
angles ~ 1 to 3 degrees 

Motor System: 

Bandwidths: 1 to 10 Hz 

Active Touch with All Three Systems 
JNDs for two-fingered pinch grasp: 

Length ~ 10% or less 
Force ~ 7% 

Compliance ~ 8% 

Viscosity ~ 14% 

Mass ~ 21% 


Figure 15 
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RESEARCH NEEDS 


A comprehensive program to develop a variety of haptic interfaces for virtual environments and 
teleoperation needs to include research in the areas shown below. Since progress in the three areas is 
interdependent, the desirable course of development for a challenging application is to continually build 
improved versions of haptic devices based on experimental data obtained from the previous versions on 
the performance of humans, devices and the interaction between the two. 


• Human Haptics 

Biomechanics 

Psychophysics 

• Hardware 

Position trackers 
Force displays 
Tactile displays 

• Software 

Multimodal interactions 
Real time simulations 

• Matching Human and Device Performance 

Comfort 

Simulation methods 
Evaluation 


Figure 16 
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